{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**HoloViews encapsulates your continuous or discrete data into storable, composable, sliceable objects, leveraging matplotlib or Bokeh for visualization and IPython/Jupyter Notebook for maximum productivity.**\n",
    "\n",
    "Main features:\n",
    "\n",
    "* Preserves your raw data while allowing rapid, interactive exploratory analysis.\n",
    "* Builds even complex, composite visualizations that are easily customized.\n",
    "* Succinct, declarative style allows you to keep all the necessary code visible, ensuring your workflow is transparent and fully reproducible.\n",
    "* You can select, slice, or transform only the relevant data that is currently of interest.\n",
    "* You can store your raw data as HoloViews objects via pickling, for later analysis or visualization even without the code that generated it.\n",
    "* Strong support for the IPython/Jupyter Notebook, including tab-completion throughout and convenient IPython magics (with all functionality available from pure Python as well).\n",
    "* Seamless (optional) interaction with Pandas Dataframes.\n",
    "* [And much, much more!](http://holoviews.org/features.html)\n",
    "\n",
    "The [IPython/Jupyter notebook](http://ipython.org/notebook/) environment and [matplotlib](http://matplotlib.org) or [bokeh](http://bokeh.pydata.org) allow you to do interactive exploration and analysis of your data and measurements, using the rich [ecosystem of tools available in Python](http://scipy.org).  However, your notebooks can very quickly fill up with verbose, specialized plotting code whenever you want to visualize your data, which is often.  To make all this practical, you can use HoloViews to greatly improve your productivity, requiring orders of magnitude fewer lines of code and letting you focus on your data itself, not on writing code to visualize and display it.\n",
    "\n",
    "Here we will show some of the main features of HoloViews, focusing on why it is useful for scientists and engineers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import holoviews as hv\n",
    "import numpy as np\n",
    "hv.notebook_extension()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A simple function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, let us define a mathematical function to explore, using the Numpy array library:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def sine(x, phase=0, freq=100):\n",
    "    return np.sin((freq * x + phase))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will examine the effect of varying phase and frequency:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "phases = np.linspace(0,2*np.pi,11) # Explored phases\n",
    "freqs = np.linspace(50,150,5)      # Explored frequencies"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Over a specific spatial area, sampled on a grid:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dist = np.linspace(-0.5,0.5,202)   # Linear spatial sampling\n",
    "x,y = np.meshgrid(dist, dist)\n",
    "grid = (x**2+y**2)                 # 2D spatial sampling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Compact data visualization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With HoloViews, we can immediately view our simple function by creating ``Image`` and ``Curve`` objects to visualize the 2D arrays and 1D cross-sections generated by the ``sine`` function, respectively:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "freq1 = hv.Image(sine(grid, freq=50))  + hv.Curve(zip(dist, sine(dist**2, freq=50)))\n",
    "freq2 = hv.Image(sine(grid, freq=200)) + hv.Curve(zip(dist, sine(dist**2, freq=200)))\n",
    "(freq1 + freq2).cols(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, ``Image`` takes a 2D numpy array as input and ``Curve`` accepts a list of ``(x,y)`` points. With the ``+`` operator we can lay these elements out together and with ``.cols(2)`` we can arrange them into two columns.\n",
    "\n",
    "HoloViews objects like ``Image`` and ``Curve`` are a great way to work with your data, because they display so easily and flexibly, yet preserve the raw data (the Numpy array in this case) in the ``.data`` attribute. [Calling matplotlib directly](http://matplotlib.org/examples/pylab_examples/multi_image.html) to generate such a figure would take much, much more code, e.g., to label each of the axes, to create a figure with subfigures, etc.  Moreover, such code would be focused on the plotting, whereas with HoloViews you can focus directly on what matters: your data, letting it plot itself.  Because the HoloViews code is so succinct, you don't need to hide it away in some difficult-to-maintain external script; you can simply type what you want to see right in the notebook, changing it at will and being able to come back to your analysis just as you left it.\n",
    "\n",
    "Only two element types are shown above, but HoloViews supports many other types of element that behave in the same way: **scatter points, histograms, tables, vectorfields, RGB images, 3D plots, annotations**, and ***many more*** as shown in the [Elements overview](Elements.ipynb).  All of these can be combined easily to create even quite complex plots with a minimum of code to write or maintain."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Interactive exploration"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can interactively explore this simple function if we declare the dimensions of the parameter space to be explored (``dimensions``) as well as the specific samples to take in this parameter space (``keys``):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dimensions = ['Phase', 'Frequency']\n",
    "keys = [(p,f) for p in phases for f in freqs]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we create a high-dimensional [``HoloMap``](Exploring_Data.ipynb) to explore: The tuple keys are the points in the parameter space, and the values are the corresponding ``Image`` objects:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "items = [(k, hv.Image(sine(grid, *k), vdims=['Amplitude'])) for k in keys]\n",
    "circular_wave = hv.HoloMap(items, kdims=dimensions)\n",
    "circular_wave"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can still compose as many visualization elements as you wish together. Here is a demonstration of how to generate the horizontal cross-section of the circular wave using ``Curve`` elements. This is then positioned next to our circular wave:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "items = [(k, hv.Curve(zip(dist, sine(dist**2, *k)))) for k in keys]\n",
    "sections = hv.HoloMap(items, kdims=dimensions)\n",
    "circular_wave + sections"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can then easily [export](Exporting.ipynb) your ``HoloMap`` objects to an interactive notebook, video formats, or GIF animations to use on a web page."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Succinct Analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "HoloViews is focused on making your data readily available, both for visualization and for numerical analysis.  Because HoloViews objects preserve your raw data, you can use any Python analysis tools you wish, such as those in the [SciPy](http://scipy.org) library.  We also support some very general-purpose data analysis and manipulation operations directly in HoloViews, exploiting the generality of the HoloViews data structures to simplify common tasks. \n",
    "\n",
    "Here, we pick a point on the circular wave at ``(0,0.25)`` and plot the amplitude value as a function of phase. The circular wave is shown annotated with a point at the chosen position. Note how we can still interactively explore the remaining dimensions, namely ``Frequency``."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sample_pos = (0,0.25)\n",
    "annotated = circular_wave * hv.Points([sample_pos])\n",
    "sample = circular_wave.sample(samples=[sample_pos]).to.curve('Phase', 'Amplitude', ['Frequency'])\n",
    "annotated + sample"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the spatial frequency of our curve plot is *not* affected by the frequency of our wave. That is because the phase always spans exactly one cycle at any of the chosen frequencies."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is the circular wave annotated with contours at the 0.5 level, followed by a thresholded version of the same wave, and then the gradient of this pattern, along with a histogram of the gradient values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%output holomap='gif'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that GIF support requires [ImageMagick](http://www.imagemagick.org/) which is installed by default on many Linux installations and may be installed on OSX using [brew](http://brew.sh/). For more information on how to install ImageMagick (including Windows instructions) see the [installation page](http://www.imagemagick.org/script/binary-releases.php)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%opts Image (cmap='gray') Contours (color='r')\n",
    "from holoviews.operation import contours, threshold, gradient\n",
    "m = hv.HoloMap([(p, hv.Image(sine(grid, phase=p))) for p in phases], kdims=['Phase'])\n",
    "contours(m, levels=[0.5]) + threshold(m, level=0.5) + gradient(m).hist(bin_range=(0,0.7))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(The ``%%`` syntax for specifying options will be briefly explained in the next section below.)  \n",
    "\n",
    "You can see that this data is shown as an animation, because the data covers multiple phases; each frame of the animation shows the result from one phase.  To get a static plot, just select one phase out of this space instead of a whole list of phases as done here.  Throughout HoloViews, if there are more dimensions of data than will fit into the plot as it has been laid out, they will simply be displayed as an animation or using slider bars.  Supplied operations include **gradient, fft_power, convolve, histogram** and other common general-purpose analysis tools."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Flexible display system"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Remember the initial test of the ``sine`` function? Those two composite objects are still in the namespace and even though the objects have already been created, we can still style the display in any way we like.  This styling is done separately from specifying your data, so that your data's specification is always clearly visible, independently of how it is being viewed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%opts Image (cmap='RdYlGn') Curve (color='g')\n",
    "(freq1 + freq2).cols(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For convenience, the IPython-magic syntax for [setting options](Options.ipynb) like ``%%opts`` is used throughout these tutorials.  However, you can use pure Python code to control the options in a similar way, e.g. from within an external non-IPython program where you want to render a HoloViews plot straight to a file, though it requires a few more curly brackets and quote marks:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "red_wave = circular_wave.opts(options={'Image':{'style':{'cmap':'RdGy'}}})\n",
    "red_wave"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Understanding your data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "HoloViews is designed to make it easy to understand your data. For instance, consider two circular waves with very different amplitudes: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "comparison = hv.Image(sine(grid)) + hv.Image(sine(grid, phase=np.pi)*0.02)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "HoloViews ensures that these differences are visible by default, by normalizing across any elements of the same type that are displayed together, and even across the frames of an animation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%opts Image (cmap='gray')\n",
    "comparison = hv.Image(sine(grid)) + hv.Image(sine(grid, phase=np.pi)*0.02)\n",
    "comparison"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This default visualization makes it clear that the two patterns differ greatly in amplitude. However, it is difficult to see the structure of the low-amplitude wave in **B**.  If you wish to focus on the spatial structure rather than the amplitude, you can instruct HoloViews to normalize data in different axes separately:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%opts Image {+axiswise} (cmap='gray')\n",
    "comparison"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Similarly, you could supply ``+framewise`` to tell it to normalize data per frame of an animation, not across all frames as it does by default.  As with any other customization, you can always specify which specific element you want the customization to apply to, even in a complex multiple-subfigure layout."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Getting Started"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "HoloViews is designed as a highly modular library with only minimal dependencies, so you only need to focus on the components that implement the functionality that you actually need to use.  With that in mind, there are several different ways that HoloViews can fit into a scientific and engineering workflow.  For each approach, we point you to the relevant tutorials that will help you get started."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The easy way"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you just have some simple data in Python, such as a few dozen 1D and 2D Numpy arrays from any source, HoloViews makes it very simple to view those as images, curves, 3D surfaces, etc., and combine them into composite figures any way you like.  \n",
    "\n",
    "To use HoloViews the easy way, just work through the [Introduction](Introduction.ipynb) tutorial, then pick suitable [Elements](Elements.ipynb) for your data types, then make figures using ``+`` to lay out figures side by side, and ``*`` to overlay curves, etc. on top of each other.  You can read about changing [options](Options.ipynb) if you want, or just follow the examples in the other tutorials. You should be able to ignore the more powerful features below, while still being able to build complex figures much, much more simply and conveniently than you could using matplotlib or bokeh directly.\n",
    "\n",
    "When you are ready, you can automatically [export](Exporting.ipynb) your figures and completed notebooks to files on disk, ready for use in publications and reports."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The better way"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The easy way is fine, but you still end up with the same data you started with -- you can't easily slice, decompose, recompose, and repackage your data, because each bit of it has been wrapped into separate HoloViews Element objects.  \n",
    "\n",
    "The better way is to move all or large ranges of your data into HoloViews Container objects, organizing it in a way that is meaningful to you.  Once it is all organized, you can then slice, select, sample, and animate whatever combination of data you want to analyze at any given time, using convenient and succinct HoloViews operations, always yielding something that can be visualized directly and with no further coding.  The ``+`` and ``*`` operations are an easy way to generate some of these containers, but there are other containers that work in very different ways that are important for other visualizations and analyses, such as parameter space exploration.\n",
    "\n",
    "To set things up in the better way will take some time, because you will have to learn about which HoloViews containers are appropriate for the types of data you have and how you want to manipulate it. You should start with the [Introduction](Introduction.ipynb) tutorial above for the basics, then work through the [Exploring Data](Exploring_Data.ipynb) Tutorial to understand what sort of operations are possible on the data.  You can then see examples of each of the different [Container](Containers.ipynb) objects, along with a reference for the complete, most fully [general container structure possible](Composing_Data.ipynb) in HoloViews.\n",
    "\n",
    "Once you see how to do what you want, the better way isn't hard to use, as you can see in some of the examples above, but it will take some time at first to understand how it all works!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The best way"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The best way to use HoloViews is if someone provides you the data already wrapped up as HoloViews objects, organized into meaningful categories and dimensions according to how you work in your own field. This way, you can sit down to your data ready to do your analyses and visualizations without even thinking about code, just specifying what data you want to see, in what combination, and letting HoloViews do the rest.\n",
    "\n",
    "Exporting data in HoloViews format is very simple, and adding HoloViews export support to a program requires very little code.  In fact, it's mostly the same code as in the easy way, just done for you by a program.  For instance, all of the data generated from the [ImaGen](http://ioam.github.io/imagen) image-generation library can be viewed directly as a HoloViews ``Image`` or ``RGB`` object.  With just a few lines of code added to the base classes in the project, ImaGen objects can be viewed immediately within IPython notebook without needing any matplotlib or Bokeh programming.  Users of ImaGen don't even need to know that e.g. matplotlib exists, because as long as it is installed ImaGen objects will just automatically display properly in an IPython/Jupyter notebook.  ImaGen also exploits some of the general-purpose data structures provided by HoloViews, for its own implementation, but doing so is not required just to be able to export data back into HoloViews.\n",
    "\n",
    "A much richer interface to HoloViews is provided by the FeatureMapper library, which is mainly used as the data-analysis package for the Topographica neural simulator.  You can see the power of HoloViews in the [Topographica Tutorials](http://ioam.github.io/topographica/Tutorials/) notebooks; with very little code users can do even quite complex analyses on continuous data without having to deal with the underlying discrete array data structures.\n",
    "\n",
    "Of course, using the best way requires someone to add HoloViews support to your simulator, data acquisition, or data analysis tools, which leads to the next way."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The nuts and bolts way"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nearly all of the HoloViews tutorials focus on interactive use within the IPython Notebook interface, because the rich display system of the notebook allows you to view HoloViews objects as soon as you have defined them, with no further coding or files required.  However, nothing in HoloViews requires IPython, and so you can also use it within your pure Python programs.  \n",
    "\n",
    "First, you can easily import ``holoviews.core`` into your own program, which only has Numpy and Param as dependencies, both of which have no required dependencies. This will allow you to create and export HoloViews objects, either to save to disk for later analysis, or when called in a Python session.\n",
    "\n",
    "You can also use the full features of HoloViews in a purely non-interactive mode, without IPython notebook or any windowing systems.  I.e., you can create HoloViews objects in Python, customize them with whatever styling options you like, and then [render them directly](Options.ipynb) to a ``.png``, ``.gif``, or ``.svg`` file on disk, perhaps to serve them directly to the web as part of an automated analysis workflow.\n",
    "\n",
    "Finally, HoloViews itself is designed to be extensible.  If you want, you can [directly manipulate the matplotlib objects](https://github.com/ioam/holoviews/wiki/Using-HoloViews-without-IPython) constructed by HoloViews, e.g. to add functionality not currently offered by HoloViews.  You can also subclass or copy any existing [element](Elements.ipynb) type to change its behavior or add features you need.  Note that HoloViews is explicitly designed as a general-purpose library, focusing on visualizations and analyses common to very many different areas of research, but researchers in different fields may want to build toolboxes of additional specialized plot types suitable for their domain.  Once defined, these new visualizations will all seamlessly combine (adjoin, overlay, etc.) with existing HoloViews objects."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Have fun!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Whichever way you choose, HoloViews is designed to support your workflow, from initial exploration to final publication. HoloViews is agnostic to whatever task or data you happen to be analyzing, allowing you to discover whatever is important for your engineering applications or scientific problems of any sort. It lets you focus on your data, not on writing plotting code!\n",
    "\n",
    "To learn more, check out the many [other tutorials and notebooks](http://holoviews.org/Tutorials), including:\n",
    "\n",
    "* [Introduction](Introduction.ipynb): Step-by-step explanation of the basic concepts in HoloViews\n",
    "\n",
    "* [Exploring Data](Exploring_Data.ipynb): How to use HoloViews to explore heterogenous collections of data, by combining and selecting your data of interest\n",
    "\n",
    "* [Options](Options.ipynb): How to find out all of the options available for a given component, and set them from within Python or IPython\n",
    "\n",
    "* [Elements](Elements.ipynb): Overview of all the basic ``Elements``\n",
    "\n",
    "* [Containers](Containers.ipynb): Overview of all the containers for ``Elements``"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python",
   "pygments_lexer": "ipython3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
